n64split v0.4a: ROM splitter and build system 1 2 3 4 5 »
Users browsing this thread: shygoo

Edit | Delete | Reply | Report | Link
n64split - ROM splitter, texture ripper, disassembler, script decoder
n64split is a tool capable of splitting an N64 ROM into its human editable assets: textures, assembly code, level data, geometry, music, and models; and include a build system that is able to take changes made to those assets and reconstruct a ROM.  The resulting ROM is almost bit compatible with the stock ROM and is N64 console compatible.

​Windows Download v0.4a
​Source Code and Project Page

Changelog
Spoiler :

​Update 19 Dezember 2015: n64split v0.4a

  • ​new config files: SM64 (J), SM64 (E), MK64 (U)
  • ​add IA4 and IA1 graphics types
  • ​add command line options: large textures, proc table, output dir
  • ​disassemble %hi/%lo macros and BAL instructions
  • ​add automatic config file detection based on header checksum
  • ​corrected a lot of textures in the SM64 configs
  • ​add music sequence bank and instrument set parsing
  • ​add collision data parser and export to Wavefront OBJ
  • ​add sm64walk tool to walk through SM64 level scripts
  • ​correct SM64 behavior script lengths

​Update 13 July 2015: n64split v0.3.1a

  • ​fix major bug introduced in n64graphics due to texture renaming
  • ​update INFO and stats messages from n64split
  • ​add ROM validation and support for .v64 (BADC ordered) ROMs
  • ​add hello world example

​Update 10 July 2015: n64split v0.3a

  • ​100% of behavior scripts decoded and most behavior references detected
  • ​~99% of asm sections disassembled
  • ​use label for references in geo layout commands 0x0A, 0x0E, 0x0F, 0x12, 0x18, 0x19, 0x1C
  • ​generate assembly files for standalone geo layout sections
  • ​detect more LA pseudo-ops that are split across JALs
  • ​add option to configure number of columns in data ptr tables
  • ​decode interaction table and functions
  • ​decode cut scene tables and functions
  • ​decode camera change table and functions
  • ​decode camera preset tables and functions
  • ​automatic detection and disassembly of dummy stub functions
  • ​removed individual texture subdirectories
  • ​add examples to skip startup screens and start right in level
  • ​switch HUD toggle example based off of Kaze's method from Green Stars

Usage
You can either drag-and-drop the SM64 ROM on the "split.bat" file
[Image: n64split_0.4a.png]
or run it from the command line:
​tools\n64split "Super Mario 64 (U).z64"

This will split the ROM based on the contents of the configuration file "sm64.u.config".  The config file is basically a memory map that controls how the ROM will be split.  The default output directory is "sm64.split" which includes sm64.s top-level assembly file, behavior file, and directories for the levels, geo layout, textures, models and music:

​sm64.split/
+- bin/ - raw binary data from undecoded sections and MIO0 blocks
+- geo/ - decoded geo layout data
+- levels/ - decoded level data
+- models/ - level and collision models
+- music/ - M64 music files
+- textures/ - all ripped textures
+- behavior_data.s - behavior command bank
+- sm64.s - top level assembly
+- Makefile - generated Makefile with texture and level dependencies


Build System
The build system isn't very useful at the moment unless you just want to replace ASM, textures, or behaviors.  It does not yet support level importing unless you are brave enough to do it manually. Music importing is better, but still requires some manual tweaks.

The build system relies on GNU make (included) to detect changes in resources and mips64-elf assembler and linker to rebuild the ROM.  You'll need to either build or obtain mips64-elf toolchain on your own if you want to rebuilt the image.  The developer of cen64 also makes n64chain which provides prebuilt mips64 toolchains.  The current list is below, but you can find more info on his website: http://git.cen64.com/?p=n64chain.git

​Usage
With mips64-elf bin directory in your PATH, just run 'make' from the command line, and it will handle the rest.
​> make
mips64-elf-as -mtune=vr4300 -march=vr4300 -o build/sm64.o gen/sm64.s
mips64-elf-ld -Tn64.ld -Map build/sm64.map -o build/sm64.elf build/sm64.o
mips64-elf-objcopy build/sm64.elf build/sm64.bin -O binary
./tools/n64cksum build/sm64.bin sm64.gen.z64


​Note: you will see warnings about "multiple instructions in a branch delay slot" but those can be safely ignored.

Examples
I've included a set of simple examples of things you can do.  View the README in the examples directory on usage, but here are some screenshots, because who doesn't like pictures?

​Hello World!
[Image: 9ppWwV3.gif]
Thanks to Jedi for implementing this version and Kaze for his ASM Tutorial!

​Texture Replacement
[Image: snap0002_sm.jpg]
Wow, I have an autographed copy of SM64!

​Skybox
[Image: skybox_sm.jpg]
Do a barrel roll!

​Color coins
[Image: color_coins_sm.jpg]
Thanks to info from cpuHacka101!

​HUD Toggle with R
[Image: hud_toggle_sm.3.jpg]
Thanks to Skelux and Kaze for documenting their HUD toggle code.

Details
Spoiler :

High level n64split flow:
[Image: n64split_sm.3.png]

High level build system architecture:
[Image: n64buildsystem_sm.3.png]

​Texture Ripping
The textures are ripped into proper PNG files with alpha channels in the gen/textures/ directory.  All of the textures are ripped including unused ones. Current texture count is 1662 (1096 if just counting merged skyboxes).

​Disassembly
The disassembler is recursive, so it will disassemble the procedures in the labels list and all the ones it discovers through JALs.  It also somewhat intelligently decodes instruction pairs to macro instructions (like LUI/ADDIU to LA and LUI/LW pairs to LW) and adds proper labels JALs and local branches.  I still need to add the same for SW so we can get the $at register back.  e.g., here is the disassembly it produces for LevelCommand_05:


Code:
​LevelCommand_05: # begin 8037E580 (0FB300)
 addiu $sp, $sp, -0x18
 sw    $ra, 0x14($sp)
 lw    $t6, 0x8038be28 # lui $t6, 0x8039/lw $t6, -0x41d8($t6)
 jal   SegmentedToVirtual
 lw    $a0, 4($t6)
 lui   $at, 0x8039
 sw    $v0, -0x41d8($at)
 b     .LLevelCommand_05_28
 nop
.LLevelCommand_05_28: # 8037E5A8
 lw    $ra, 0x14($sp)
 addiu $sp, $sp, 0x18
 jr    $ra
 nop


Tools
In the tools directory, there are some other misc tools that you might find useful.  They will print built-in help if you run them without any options on the command line.


  • ​f3d - tool to decode Fast3D display lists
  • ​mio0 - standalone MIO0 compressor/decompressor
  • ​n64cksum - standalone N64 checksum generator.  can either do in place or output to a new file
  • ​n64graphics - converts graphics data from PNG files into RGBA or IA N64 graphics data
  • ​mipsdisasm - standalone recursive MIPS disassembler
  • ​sm64geo - standalone SM64 geometry layout decoder
  • ​sm64extend - fast, compatible SM64 ROM extender
  • ​sm64compress - SM64 extended ROM fixer and compressor
  • ​sm64walk - SM64 level script interpreter


There may be bugs and you can probably get it to crash.  Feel free to report any issues on github or here.

Thanks to Skelux, Kaze, cpuHacka101, shygoo, David, and Jedi for inspiration, examples, and putting up with all my stupid questions.
(This post was last modified: December 19th, 2015, 04:47 am by queueRAM. Edit Reason: Update screenshot )

Edit | Restore | Reply | Report | Link
This looks pretty essential, might want to hide it from the le machinimists Wink
Gone for a while.

Edit | Delete | Reply | Report | Link
Update 31 May 2015: n64split 0.2a release

  • ​Decode a lot more LUI/ADDI to LI or LA pseudo instructions
  • ​Add geometry section type
  • ​Properly detect 0x11 and 0x1D variable length geometry commands
  • ​Decode and relocate all level scripts starting at top-level
  • ​Print ROM decoding statistics
  • ​Add level, geo, and behavior jump tables
  • ​Generate linker script from config file data
  • ​Add full behavior bank decoding
  • ​Add standalone geometry layout decoding tool: sm64geo
  • ​Add behavior modification example


Download Link:
​Windows Download v0.2a
​Source Code and Project Page

Edit | Delete | Reply | Report | Link
Finally someone makes a split disassembly of Super Mario 64.

Edit | Delete | Reply | Report | Link
I should have said so earlier, but I don't have enough praise for this project! I can tell a LOT of work went into it.

In my humble opinion, anyone who is trying to fill the gaps in SM64 hacking knowledge without using this is doing it very wrong! The transition from binary to .s alone makes hacking so much more pleasant, because you can add whitespace and #comments, as well as #comment things out that you're trying to replace.

The #1 feature that I want to see added very badly (and I don't care how it's implemented) is the ability for the suite to operate on its output. The goal being that any hacks made with early versions of n64split could be split with later versions. How doable would that be?

Other than that (and perhaps as a means toward that) I'd like to see it optionally operate-on and produce extended ROM's. In addition to increasing cross-tool compatibility, I hope that skipping the MIO0 compressor will speed up the development cycle in most cases.

I realize these aren't trivial requests, so I'm willing to work on them if they're not a priority for you. I suspect there's some overlap with the work I need to do to fully parse levels in my editor anyway.

Thanks again for making this wonderful software, as well as releasing the source [under the MIT license, no less]!

Edit | Delete | Reply | Report | Link
(July 6th, 2015, 12:40 am)Jedi Wrote: The #1 feature that I want to see added very badly (and I don't care how it's implemented) is the ability for the suite to operate on its output. The goal being that any hacks made with early versions of n64split could be split with later versions. How doable would that be?

Thanks for the feedback!  You got me thinking about this when you asked about splitting existing extended ROM hacks.  Under the current implementation, you'd have to modify the config file to point at the extended ROM's addresses for the level, geo, and texture data.  I think I could redo to the SM64-specific data parsing to have it more intelligently walk the level and geo scripts, but it'd still need guidance on texture addresses.  I haven't learned enough to extract those automatically.  I actually manually found all of the F3D references to the textures and added them to the config file by hand.  Manual listing in the config file also has its benefits though: providing a reference memory map and allowing me to enter unused textures and geo layout data.  Maybe I can find a way to support both.

(July 6th, 2015, 12:40 am)Jedi Wrote:
Other than that (and perhaps as a means toward that) I'd like to see it optionally operate-on and produce extended ROM's. In addition to increasing cross-tool compatibility, I hope that skipping the MIO0 compressor will speed up the development cycle in most cases.

This is a topic that I'm divided on.  If we skip the MIO0 compressor, then we'd have to change level scripts and/or level parsing functions to handle the uncompressed data somewhat similar to how the SM64 ROM extender does it.  I initially wanted to create a splitter and build system that could regenerate an exact copy of the stock ROM, and provided code patches to extend it in other ways.  Changing the data during the split would break this idea.  However, it might be much cleaner interface to the user and the build system would actually gain some benefits from being able to use the linker for a lot more segmented address references without resorting to writing more tools.  I know shygoo has been working out a way to rewrite the level command so a fake MIO0 header doesn't need to be attached.  I could perhaps incorporate a patching system in the splitter to automatically patch the commands as it splits in version 0.4.

Edit | Delete | Reply | Report | Link
Update 10 July 2015: n64split 0.3a release

  • ​100% of behavior scripts decoded and most behavior references detected
  • ​~99% of asm sections disassembled
  • ​use label for references in geo layout commands 0x0A, 0x0E, 0x0F, 0x12, 0x18, 0x19, 0x1C
  • ​generate assembly files for standalone geo layout sections
  • ​detect more LA pseudo-ops that are split across JALs
  • ​add option to configure number of columns in data ptr tables
  • ​decode interaction table and functions
  • ​decode cut scene tables and functions
  • ​decode camera change table and functions
  • ​decode camera preset tables and functions
  • ​automatic detection and disassembly of dummy stub functions
  • ​removed individual texture subdirectories
  • ​add examples to skip startup screens and start right in level
  • ​switch HUD toggle example based off of Kaze's method from Green Stars

Download Link:
​Windows Download v0.3a
​Source Code and Project Page

Edit | Delete | Reply | Report | Link
(July 10th, 2015, 03:00 pm)queueRAM Wrote: ​Update 10 July 2015: n64split 0.3a release

  • ​100% of behavior scripts decoded and most behavior references detected
  • ​~99% of asm sections disassembled
  • ​use label for references in geo layout commands 0x0A, 0x0E, 0x0F, 0x12, 0x18, 0x19, 0x1C
  • ​generate assembly files for standalone geo layout sections
  • ​detect more LA pseudo-ops that are split across JALs
  • ​add option to configure number of columns in data ptr tables
  • ​decode interaction table and functions
  • ​decode cut scene tables and functions
  • ​decode camera change table and functions
  • ​decode camera preset tables and functions
  • ​automatic detection and disassembly of dummy stub functions
  • ​removed individual texture subdirectories
  • ​add examples to skip startup screens and start right in level
  • ​switch HUD toggle example based off of Kaze's method from Green Stars

Download Link:
​Windows Download v0.3a
​Source Code and Project Page


Cool b3ans. I think this is gonna be really useful, hoho. Thx.
R.I.P Tarek701. 2005-2016

Edit | Delete | Reply | Report | Link
Does this tool work with more than SM64?
DSHack.org
Spoiler :

[Image: loadcard.php?id=1996]


Edit | Delete | Reply | Report | Link
(July 12th, 2015, 05:52 pm)Solaris Wrote: Does this tool work with more than SM64?


In theory n64split can be configured to work with any N64 ROM.  However, each game (and each version of each game) will need its own config file spelled out so the tool knows how to split it up.  Also, some custom script parsers would probably need to be written for each game, such as the level, geo layout, and behavior parsers that I created for SM64.

If there is a game that you are interested in running through n64split and you have some hacking notes for it, I can help take a stab at generating an initial config file for it.

n64split v0.4a: ROM splitter and build system 1 2 3 4 5 »
Users browsing this thread: shygoo

Quick Reply




Inline Post Moderation:  

Moderation Options:


Loading...
  EN ・日本語 
Expand chat